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9.2.0 End of Chapter Problems 


Problem 1 

Let X be a continuous random variable with the following PDF 

, 6x( 1 - x) if 0 < x < 1 


f/ x ) = < 

lo 


otherwise 


Suppose that we know 


7 | X=x ~ Geometric(x). 
Find the posterior density of X given Y = 2,f X \ y( x 12 ). 


Problem 2 

Let X be a continuous random variable with the following PDF 

i 3x 2 if 0 < x < 1 


M x ) = { 

lo 


otherwise 


Also, suppose that 

Y \X=x 

Find the MAP estimate of X given 7=5. 


Geometric(x). 


Problem 3 

Let X and 7 be two jointly continuous random variables with joint PDF 
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(x+2_y 2 o < x, v < 1 

fxv( x >y) = \ 

\ 0 otherwise. 

Find the MAP and the ML estimates of X given Y = y. 


Problem 4 

Let X be a continuous random variable with the following PDF 


fjfa) = 


2x 2 + r if 0 < x < 1 


otherwise 


We also know that 


/y|xC v l x ) 



0 


if 0 < y < 1 

otherwise 


Find the MMSE estimate ofX, given Y = y is observed. 


Problem 5 

LetX ~ N(0, 1) and 

Y= 2X+ W, 

where W ~ N( 0, 1) is independent ofX. 

a. Find the MMSE estimator of A given Y, (X M ). 

A 

b. Find the MSE of this estimator, using MSE = E[(X - X M ) 2 \. 
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c. Check that E\X~\ = AjA^] + is [A 2 ]. 


Problem 6 

1 

Suppose A ~ Uniform(0, 1), and given A = x,Y ~ Exponential(X = —). 

a. Find the linear MMSE estimate of A given Y. 

b. Find the MSE of this estimator. 

c. Check that E[XY] = 0. 


Problem 7 

2 

Suppose that the signal X ~ N{ 0, ay) is transmitted over a communication channel. Assume 
that the received signal is given by 


Y = X+ W, 

2 

where W ~ MO, a w ) is independent of A. 

a. Find the MMSE estimator of Agiven Y, (X M ). 

b. Find the MSE of this estimator. 


Problem 8 

Let Abe an unobserved random variable with EX = 0, Var(A) = 5. Assume that we have 
observed Y ] and Y 2 given by 


Y i = 2X+ W 1? 

y 2 =x+w 2 , 

where EFY l = EW 2 = 0, Var( IV ^) = 2, and Var( IV 2 ) = 5. Assume that W { , W 2 , and Aare 
independent random variables. Find the linear MMSE estimator of A, given F| and Y 2 . 
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Problem 9 

Consider again Problem 8 . in which A is an unobserved random variable with EX = 0, 

Var(A) = 5. Assume that we have observed Y l and Y 2 given by 

Y x = 2A + W { , 

y 2 =x+w 2 , 

where EW X = EW 2 = 0, Var( tfj) = 2, and Var( IV 2 ) = 5. Assume that W x , W 2 , and A are 
independent random variables. Find the linear MMSE estimator of A, given Y x and F 9 , using 
the vector formula 


X L = C XY C Y l (Y-E[Y]) + E[X\. 


Problem 10 

Let Abe an unobserved random variable with EX = 0, Var(A) = 5. Assume that we have 
observed Y x , Y 2 , and 7 3 given by 


F, = 2A+ W v 

y 2 = x+w 2 , 

Y 3 =X+2W 3 , 

where EW { = EW 2 = EW 3 = 0, Var (WJ = 2, Var(tf 2 ) = 5, and Var(lV 3 ) = 3. Assume that W { 
, W 2 , W 3 , and A are independent random variables. Find the linear MMSE estimator of A, 
given F|, Y 2 , and Y 3 . 


Problem 11 


Consider two random variables A and Y with the joint PMF given by the table below. 



T! 

II 

o 

Y = 1 

1 

o 

II 

1 

3 


7 

7 
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1 



II 

3 

7 

0 


a. Find the linear MMSE estimator of Agiven Y, (X L ). 

b. Find the MMSE estimator of Agiven Y, (X M ). 

c. Find the MSE ofX M . 


Problem 12 


Consider two random variables X and Y with the joint PMF given by the table below. 



o 

II 

fc-H 

Y = 1 

II 


1 

l 


o 

II 

* 

6 

3 

0 


1 


1 

A = 1 

— 

0 

— 


3 


6 


a. Find the linear MMSE estimator of X given Y, (X L ). 

b. Find the MSE of A^. 

c. Find the MMSE estimator of X given Y, (X M ). 

d. Find the MSE ofX M . 


Problem 13 

Suppose that the random variable X is transmitted over a communication channel. Assume that 
the received signal is given by 


Y= 2X+ W, 

where W ~ N{ 0, a 2 ) is independent of A. Suppose that X = 1 with probability p, and A = - 1 
with probability 1 - p. The goal is to decide between A = - 1 and X- 1 by observing the 
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random variable Y. Find the MAP test for this problem. 


Problem 14 

Find the average error probability in Problem 13 . 


Problem 15 

A monitoring system is in charge of detecting malfunctioning machinery in a facility. There 
are two hypotheses to choose from: 

H q : There is not a malfunction, 

Hy : There is a malfunction. 

The system notifies a maintenance team if it accepts Hy. Suppose that, after processing the 
data, we obtain P(Hy \y) = 0.10. Also, assume that the cost of missing a malfunction is 30 
times the cost of a false alarm. Should the system alert a maintenance team (accept H^)l 


Problem 16 


1 

LetXand Y be jointly normal andX ~ N( 2, 1), Y ~ N(l, 5), and p(X, Y) = Find a 90% 
credible interval forX, given Y = 1 is observed. 


Problem 17 

When the choice of a prior distribution is subjective, it is often advantageous to choose a prior 
distribution that will result in a posterior distribution of the same distributional family. When 
the prior and posterior distributions share the same distributional family, they are called 
conjugate distributions, and the prior is called a conjugate prior. Conjugate priors are used out 
of ease because they always result in a closed form posterior distribution. One example of this 
is to use a gamma prior for Poisson distributed data. Assume our data Y given X is distributed 
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Y | X = x ~ Poisson{X = x) and we chose the prior to be X ~ Gamma(a, fi). Then the PMF 
for our data is 


e ~ x x y 

PY\j^y\ x ) = ——> forx > G {0, 1,2, 

and the PDF of the prior is given by 

p a x a ~\ e -P x 

fjfc)= -- , forx > 0, aj > 0. 

a. Show that the posterior distribution is Gamma(a + y, [:> + 1). 

{Hint: Remove all the terms not containing x by putting them into some normalizing 
constant, c, and noting that f x \y( x \y) x Py\Js/Y I x )fx( x )-) 

b. Write out the PDF for the posterior distribution,/^ y(x\y). 

c. Find mean and variance of the posterior distribution, E[X\ Y] and Var(X| Y). 


Problem 18 

Assume our data Y given X is distributed Y \ X= x ~ Binomial(n,p = x) and we chose the 
prior to be X ~ Beta(a, f:l). Then the PMF for our data is 

P Y\dy\ x ) = ( v ) x ' V ( l _x )" ~ V , forx G [0, 1], v G {0, 1, 

and the PDF of the prior is given by 

T{a +f$) 

U x ) = Y{ a )T{P) Xa ^ ~ X) j ’ for0 - x - !»« > > °- 


a a[S 

Note that, EX = —— and Var(A) = -r-. 

a+P v 7 {a+P) 2 {a+p+ 1) 

a. Show that the posterior distribution is Beta{a + y,fi + n~ y). 

b. Write out the PDF for the posterior distribution,/^ /x y). 

c. Find mean and variance of the posterior distribution, E[X\ Y] and Var(A Y). 


Problem 19 
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Assume our data Y given X is distributed Y \ X = x ~ Geometricip = x) and we chose the 
prior to be X ~ Beta{a, [i). Refer to Problem 18 for the PDF and moments of the Beta 
distribution. 

a. Show that the posterior distribution is Beta(a + \,/3 +y - 1). 

b. Write out the PDF for the posterior distribution, f X \ } .fx y). 

c. Find mean and variance of the posterior distribution, E[X\ Y] and Var(A Y). 


Problem 20 

T 

Assume our data Y = (y l ,y 2 , ■■■,y„) given X is independently identically distributed, 
i .i .d. 

Y | X = x ~ Exponential (/l = x), and we chose the prior to be A ~ Gamma(a, fi). 

a. Find the likelihood of the function, L(Y;X) y , Y |^C v l’k2’ ■•■■hi l x )- 

b. Using the likelihood function of the data, show that the posterior distribution is 
Gammaia + /?,/>' + X/ = 

c. Write out the PDF for the posterior distribution,/^ Y (x \ y). 

d. Find mean and variance of the posterior distribution, E[X\ Y] and Var(A Y). 


previous 
next —> 
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